The next generation of satellite-based remote sensing instruments will produce an unprecedented volume of data. Imaging spectrometers, also known as hyper-spectral imagers, are prime examples. They collect image data in hundreds of spectral bands simultaneously from the near ultraviolet to the short wave infrared, and are capable of providing direct identification of surface materials.
Hyper-spectral data thus collected are typically in the form of a three-dimensional (3D) data cube. Each data cube has two dimensions in the spatial domain defining a rectangular plane of image pixels, and a third dimension in the spectral domain defining radiance levels of multiple spectral bands per each image pixel. The volume and complexity of hyper-spectral data present a significant challenge to conventional transmission and image analysis methods.
Data compression using Vector Quantisation (VQ) has received much attention because of its promise of high compression ratio and relatively simple structure. The VQ procedure is known to have two main steps: codebook generation and codevector matching. VQ can be viewed as mapping a large set of vectors into a small set of indexed codevectors forming a codebook. During encoding, a search through a codebook is performed to find a best codevector to express each input vector. The index or address of the selected codevector in the codebook is stored associated with the input vector or the input vector location. Given two systems having a same codebook, transmission of the index to a decoder over a communication channel from the first system to the second other system allows a decoder within the second other system to retrieve the same codevector from an identical codebook. This results in a reconstructed approximation of the corresponding input vector. Compression is thus obtained by transmitting the index of the codevector rather the codevector itself.
In an article entitled “Lossy Compression of Hyperspectral Data Using Vector Quantization” by Michael Ryan and John Arnold in the journal Remote Sens. Environ., Elsevier Science Inc., New York, N.Y., 1997, Vol. 61, pp. 419-436, an overview of known general vector quantization techniques is presented. The article is herein incorporated by reference. In particular, the authors describe issues such as distortion measures and classification issues arising from lossy compression of hyper-spectral data using vector quantization.
Two innovative data compression techniques have been recently disclosed by the same inventor in U.S. patent application Ser. No. 09/717,220 filed 22 Nov. 2000 issued as U.S. Pat. No. 6,701,021 and in U.S. patent application Ser. No. 09/725,370 filed 24 Nov. 2000 issued as U.S. Pat No. 6,724,940, which are incorporated herein by reference. Both compression techniques—Successive Approximation Multi-Stage Vector Quantization (SAMVQ) and Hierarchical Self-Organizing Cluster Vector Quantization (HSOCVQ), respectively—are capable to provide near-lossless data compression if a fidelity threshold parameter is set to a value such that the error induced during the compression is at the same level of the intrinsic noise of the original data.
However, implementation of a lossy compression method for real-time data compression of a continuous data flow is substantially complicated due to the fact that the complete hyper-spectral data cube is not available for compression. In real-time compression onboard a satellite, hyper-spectral data corresponding to only a 2D focal plane frame sensed at a given moment from a swath target—across track line—on ground is available together with the hyper-spectral data corresponding to 2D focal plane frames sensed before. One—spatial-dimension of the 2D focal plane frame corresponds to a line of ground samples—called ground pixels, and another dimension of the 2D focal plane frame corresponds to a spectrum expansion of each ground pixel in wavelength. The spectrum expansion of a ground pixel is referred to as a “spectral vector”. A focal plane frame comprises a same number of spectral vectors and ground pixels. The second spatial dimension of the hyper-spectral data cube is obtained by sensing successive swath targets in along-track direction of the moving satellite producing successive 2D focal plane frames.
Therefore, it is only possible to apply the compression to successive 2D focal plane frames or successive regions comprising several 2D focal plane frames substantially inhibiting successful application of lossy compression at high compression ratios. Application of conventional lossy compression methods on a region-by-region basis results in visible artifacts at the boundaries between the regions severely affecting image quality after decompression.
Furthermore, for real-time compression of a continuous hyper-spectral data flow, it is necessary to increase data throughput by using parallel operation of a plurality of compression engines. Therefore, a regional data cube is split into a plurality of smaller regional sub-cubes, referred to as vignettes herein. However, when a regional data cube is split into vignettes and each vignette is processed independently, a spatial boundary is introduced between two adjacent vignettes resulting in visible artifacts after decompression.
In U.S. patent application Ser. No. 10/606,761 filed Jun. 27, 2003 issued as U.S. Pat. No. 6,798,360 and U.S. patent application Ser. No. 10/611,897 filed Jul. 03, 2003 published as U.S. Patent Pub. No. 20050002584, which are incorporated herein by reference, the same inventor discloses compression techniques for compressing a continuous data flow in real time also enabling data compression using parallel operation of a plurality of compression engines. These innovative techniques overcome the above problems of real time data compression and allow implementation of SAMVQ as well as HSOCVQ.
Therefore, these techniques support a successful realization of a real-time wideband compression system.